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The Impact of Educational 
Technology on Student Achievement: 
Assessment of and for Learning 

The author explores current efforts by educators and policy makers to harness 
the power of educational technology for both assessment of learning and 
assessment for learning in K-12 classrooms. 


In this era of accountability brought 
about through international compari- 
sons that pit the U.S . against other na- 
tions and testing requirements resulting 
from legislation such as No Child Left 
Behind (U.S . Department of Education 
(USDOE), 2001) educators, adminis- 
trators and policy makers require more 
efficient and effective ways to measure 
and analyze student achievement data. 
Moreover, individuals at the local 
and state levels look to teachers and 
district leaders to use these analyses to 
benefit future outcomes. The pressure 
for accountability affects states and 
districts as well as classroom teachers. 
This article explores current efforts by 
educators andpolicy makers to harness 
the power of educational technol- 
ogy for both assessment of learning 
and assessment for learning in K-12 
classrooms. The first section describes 
efforts to utilize technology for student 
achievement testing, an assessment 
of learning. Issues addressed include 
high-stakes and low-stakes testing, 
issues for classroom implementation, 
test preparation programs, special 
education populations, and adaptive 
testing. The second section outlines 
the responses of states, districts and 
schools to the accountability pres- 


sures related to data storage, analysis, 
reporting and data informed decision 
making. The last section focuses on 
the role of technology in networked 
science and mathematics classrooms 
where immediate feedback devices 
provide teachers with formative as- 
sessment information about student 
learning to guide instructional strate- 
gies. In a connected classroom, the 
assessment intends to help students 
learn. The critical difference between 
assessment of learning and assessment 
for learning lies at the heart of current 
educational technology use in science 
classrooms. 

Educational Technology 
for Testing 

The increased testing requirements 
of the “No Child Left Behind” Act 
(2001) resulted in serious efforts to 
develop statewide computer-based 
testing programs to assess student 
learning. As of the 2004-2005 school 
year, 16 states have statewide com- 
puter-based testing programs in place 
while 4 additional states are piloting 
these programs (Fox, 2005). The rapid 
response, prompt retake possibilities 
for students close to the cut-off scores 
and easily captured data provided by 



The critical difference 
between assessment of 
learning and assessment for 
learning lies at the heart of 
current educational 
technology use in science 
classrooms. 

computer-based testing programs 
provide school districts with several 
advantages. On apractical level, these 
programs allow schools to gather data 
to meet the requirements of federal 
mandates. More importantly they may 
provide educators necessary informa- 
tion to individualize learning plans 
for students. Unlike their more tradi- 
tional predecessors that often require 
extensive time for scoring and data 
distribution, computer-based tests pro- 
vide immediate feedback on student 
achievement. 

However, statewide computer- 
based testing comes with a cost. While 
technology can streamline assessment 
and provide needed information for 
documentation related to accountabili- 
ty requirements , schools need comput- 
ers for students to take electronic tests. 
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Budget deficits threaten to trump these 
initiatives. In addition to hardware and 
software costs, school systems must 
wrestle with issues of test security to 
limit potential for invalidating scores 
on high-stakes tests. Students across 
the state must take the test in a limited 
time frame and with similar testing 
conditions to ensure the fairness of 
the test. Variations in equipment 
across schools and districts further 
complicate the issue. Comparisons 
of test results from paper and pencil 
tests versus computer-based tests on 
modern or out-dated equipment in- 
troduce important variables into the 
evaluation process (Olson, 2003). As 
a natural consequence of these factors, 
some states have developed online test 
preparation programs to help improve 
student scores. As of 2003, twelve 
states already had computer-based 
practice exams available to help stu- 
dent prepare for state-mandated tests 
(Borja, 2003). 

Low-stakes diagnostic computer- 
based tests offer possibilities for im- 
proving student performance without 
many of the issues related to their 
high-stakes counterparts. Some educa- 
tors believe that success on low-stakes 
tests portends improved performance 
on high-stakes tests. Further, teachers 
are the recipients of several advantages 
of classroom computerized testing 
such as decreased time grading papers 
and efficient assessment of student 
learning. Limitations of this approach 
(many of which also apply to high 
stakes testing environments) include 
the need for classroom sets of com- 
puters, difficulties with test security 
(secure sites that cannot be hacked 
into by enterprising students), and test 
sites with easy visibility of computer 
screens for wandering student eyes. 
However, as test hanks become more 
sophisticated several limitations have 
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been effectively eliminated. Teachers 
can now prepare multiple versions of 
assessment instruments for standard 
multiple choice, true-false or fill in 
the blank type questions, but questions 
that require essay style answers or 
complex multi-step problem solving 
items continue to present difficul- 
ties for computer-based testing and 
evaluation systems. The efficiencies 
of time afforded teachers by computer 
evaluation of tests are lost for these 
higher level assessment items (Gal- 
ley, 2003). 

Educational technology introduces 
both issues and advantages for mea- 
suring student achievement in special 
education classrooms. Advances 
such as assistive technologies have 
improved students with disabilities’ 
opportunities to fully participate in the 
classroom (Hitchcock, Meyer, Rose, & 
Jackson, 2002; Rose, 2001). Theuse of 
computer-based testing technologies 
extends these possibilities by intro- 
ducing tools such as spell-checkers or 
reading machines. For example, heal - 
ing impaired students in Oregon may 
select from English, American Sign 
Language or both in testing situations. 
Students with visual impairments or 
reading disabilities may take tests with 
text readers in Massachusetts, but the 
cost and limitations of text reading 
programs limits this possibility for 
general education students. Text read- 
ers still encounter problems with non 
standard symbols such as Greek letters 
or equations found in mathematics or 
science (Goldstein, 2003). 


Exploration of new technology- 
based assessment systems with special 
education populations provides educa- 
tors with insights that promise to ben- 
efit both regular and special education 
students. Indiana’s use of annual vid- 
eotapes of oral reading ability as paid 
of an electronic portfolio assessment 
represents a creative use of educational 
technology to provide a unique and 
highly individualized view of student 
achievement over a multi-year period. 
The use of these electronic portfolios 
has the added benefit of greatly reduc- 
ing the extensive paperwork needed 
to document student performance, a 
common grievance of special educa- 
tors, and provides special educators 
the opportunity to easily share IDEA- 
mandated evidence of student progress 
with parents (Goldstein, 2003). 

Another important area of research 
and test development is adaptive 
testing. In these situations, student’s 
previous performance determines sub- 
sequent questions providing educators 
with useful diagnostic information. 
Adaptive testing allows students to 
self-pace, provides a suitable challenge 
for each learner, rewards student effort 
with immediate feedback, improves 
test security issues due to the dif- 
ferentiated nature of each sequence 
of items, and offers the advantage of 
multimedia options in question design 
such as the inclusion of text, graphs, 
images, video and animations (Dun- 
kel, 1999). These forms of testing, 
however, do not meet the demands of 
NCLB to assess each student against 
state established grade level standards. 
South Dakota’s adaptive online test- 
ing program, for example, serves as a 
voluntary resource for schools while 
educators administer a paper and 
pencil test to satisfy NCLB legislation 
(Olson, 2003; Trotter, 2003). 
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Educational Technology 
and Data Management 

A critical advantage of computer- 
based testing programs is their readily 
accessible stores of data ± Teachers and 
policy makers need to access, analyze, 
and interpret student achievement data 
if these resources are to guide deci- 
sion making and strategy selection to 
improve student learning. Expensive 
and complex data management sys- 
tems focused on meeting the reporting 
requirements of the NCLB legislation 
offer the promise of easily available 
student information including de- 
mographic information for reporting 
puiposes and student achievement 
information. In a survey of state of- 
ficials, 15 states responded that NCLB 
requirements influenced decisions 
to obtain more sophisticated data 
management systems (Hoff, 2005). 
The North Carolina Window of Infor- 
mation on Student Education or NC 
WISE represents an example of these 
new data management systems and the 
challenges of developing them. The 
NC-WISE implementation program 
for this $53 million project called for 
a two-phase approach over a multi- 
year time span, which was initially 
launched in 1999 and is scheduled to 
be fully operational in 2,264 public 
schools by 2007 . When completed this 
data management system will capture 
a wide range of essential student data, 
integrate classroom, instructional and 
administrative tasks, provide informa- 
tion directly to teachers, and support 
school wide functions such as schedul- 
ing, transcript management, auditing 
and student services (Hoff, 2005 ; Reitz 
& Winter, 2004). 

Similarly, educators anticipate a 
data management tool in Ohio to aid 
in preparation of individualized edu- 
cational plans to help high-school stu- 


dents pass state mandated exit exams. 
Analysis of student’s test results will 
point to areas of student’s strengths and 
weaknesses and suggest strategies to 
help students improve their test scores. 
Another Ohio State data-management 
longitudinal project creates reports on 
individual students as they progress 
from grade 3 to grade 8 (Hoff, 2005). 
Similar projects have been proposed 
in both Arizona and Hawaii to expand 
current student data systems to im- 
prove the quality of data and teacher 
access to data (Hoff, 2005). 

The demand for more transparent 
and student centered uses of data 
about every aspect of the educational 
process has increased the necessity 
for databases that talk easily to each 
other. In the past, data tended to flow 
upstream, from local schools to dis- 
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tricts to states to the federal govern- 
ment. If schools and teachers expect 
to use these data to affect school wide 
or classroom decision making , the data 
must flow equally well in both direc- 
tions. Disparate computer systems that 
do not communicate well with each 
other interfere with this data flow. In 
addition, the wide audience for educa- 
tional data impacts choices about the 
type of data to collect and the method 
used for storage. Users representing 
varied constituencies such as the US 


Department of Education, state and 
district policy makers and educational 
agencies, school administrators, class- 
room teachers, students, parents and 
community and corporate partners 
complicate the situation due to their 
varying intended uses including meet- 
ing NCLB reporting requirements, in- 
forming parents and students regarding 
school choice, guiding administrators 
and teachers on school curriculum, 
tracking individual student progress, 
and helping each of these constituents 
determine how to invest their time and 
resources (Palaich, Good, & van der 
Ploeg, 2004). 

The type of data collected influ- 
ences its potential impact for decision 
making. Many argue that the current 
infrequent standardized high-stakes 
test data collected by states offer 
insufficient detail to guide teacher 
classroom strategy choice (Seltzer, 
Bryk, & Frank, 1994; Seltzer, Choi, 
& Thum, 2003; Wainer, Hambleton, 
& Meara, 1999). The yearly grade or 
school improvement data mandated by 
NCLB provide volatile data that do not 
serve as a reliable indicator of school 
performance (Kane & Staiger, 2002; 
Linn & Haug, 2002). In addition, older 
databases often use events such as 
enrollment dates or testing schedules 
as organizational schemes which are 
not useful for classroom level deci- 
sion making. Schools need access to 
individual student records to facilitate 
student-based decisions. Some states, 
including Michigan, Indiana, Illinois 
and Wisconsin have either already 
built statewide data bases or are in the 
process of doing so. Concerns about 
student privacy rights as spelled out 
in the Federal Educational Rights and 
Privacy Act (FERPA) (U.S. Depart- 
ment of Education, n.d.) have delayed 
this process in some cases (Palaich et 
al„ 2004). 
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In summary, growing accountabil- 
ity pressures to collect, store, analyze 
and report educational data as well as to 
generate data-based educational policy 
provide the force behind the growing 
use of statewide computer-based test- 
ing programs and sophisticated data 
management systems. 

As data needs continue to grow and 
become more complex, the standard 
issues of cost and available infrastruc- 
ture are coupled with issues of mak- 
ing a wide variety of data available 
to many different audiences. Elected 
officials, policy makers and educators 
need sophisticated technical expertise 
to grapple with the challenges of 



Connected classroom 
systems offer opportunities 
for improved formative 
assessment through 
questioning and immediate 
feedback, which has the 
potential for affording 
teachers the necessary 
information to tailor 
instruction to meet student 
needs. 

compatibility and information flow 
in designing and implementing these 
data management systems. Evolv- 
ing educational data needs require 
thoughtful solutions. 

Formative Assessment in 
Connected Classrooms 

While the present focus on stan- 
dardized testing and data collection 
and management offers both assis- 
tance and obstacles related to sup- 
porting student learning, standardized 
testing falls short of providing teachers 


with appropriate data to support daily 
decisions in the classroom. Teach- 
ers have long struggled to develop 
mechanisms for monitoring student 
progress toward understanding and 
are often frustrated by the quality of 
information gained through normal 
classroom processes. 

Some have questioned whether the 
current interest and concern in school 
accountability can lead to improved 
student achievement. These issues 
may also overshadow more complex 
matters such as educators’ efforts to 
motivate all students to want to learn 
or to increase students perceptions 
of their capability to learn (Stiggins, 
2002). While the assessment commu- 
nity focuses on ever more sophisticated 
measures of student achievement 
and works diligently to ensure reli- 
able and valid measures of student 
achievement, perhaps another worthy 
goal would be to maximize the posi- 
tive impact of test scores on student 
learning. That is, perhaps these efforts 
should be dedicated to assessment for 
learning. 

A different kind of educational 
technology use intended to support 
student achievement occurs in the 
connected classroom. Connected 
classroom technology refers to a net- 
worked system of personal comput- 
ers or handheld devices specifically 
designed to be used in a classroom 
for interactive teaching and learn- 
ing. These networked technologies 
include response systems, classroom 
communication systems, and newer 
systems included under the name 
CATAALYST (Classroom Aggrega- 
tion Technology for Activating and 
Assessing Your Students’ Thinking) 
(Roschelle, Penuel, & Abrahamson, 
2004). Connected classroom systems 
offer opportunities for improved 
formative assessment through ques- 


tioning and immediate feedback, 
which has the potential for affording 
teachers the necessary information 
to tailor instruction to meet student 
needs (Black & Wiliam, 1998; Fuchs 
& Fuchs, 1986). One mode of instruc- 
tion in connected classrooms involves 
students beaming answers to a receiv- 
ing station with an accompanying 
anonymous display of histograms of 
student answer choices. Data logs of 
student responses can be archived for 
later teacher analysis. Discourse that 
occurs in a safe environment through 
the public examination of problem 
solving and alternative conceptions 
helps students understand their role 
as critical listeners and thinkers in 
the classroom (Artzt & Yaloz-Femia, 
1999). In the connected classroom, 
teacher adaptive expertise forms the 
critical foundation for formative as- 
sessment practices that provide needed 
information to monitor student’s 
incremental progress and keep them 
oriented on the path to deep conceptual 
understanding. 

The effectiveness of the connected 
classroom technology, as with all in- 
structional tools, depends on the skill 
of the instructor. 

Formative assessment coupled with 
well orchestrated classroom discourse 
informs teacher practice by providing 
a window into both what and how 
students are thinking. Unlike sum- 
mative assessment that occurs after 
completion of instruction, formative 
assessment is ongoing and low stakes. 
The focus of this kind of assessment 
is often shifted from the right answer 
to the reasoning that led to the answer. 
In a connected classroom, students 
are informed about their classmates 
thinking processes, and students with a 
minority opinion see that they arc not 
alone. Connected classroom environ- 
ments support positive student think- 
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ing habits such as seeking alternative 
representations for problems, compar- 
ing and contrasting different solution 
strategies, and explaining and describ- 
ing problem solving strategies. These 
habits support active engagement and 
intellectual growth (Dufresne, Gerace, 
Mestre, & Leonard, 2000). 

In a traditional classroom, teachers 
collect student work, evaluate it by 
hand, and return the papers to students 
several days later (Figure 1). In many 
cases, students have forgotten the point 
of the assignment and teachers experi- 
ence difficulty correcting conceptual 
problems days after instruction on 
the topic. With the pace of instruction 
continuing unabated, student naive 
understandings accumulate resulting 
in increasing challenges for student 
achievement. In science and math- 
ematics where conceptions build in- 
crementally, delayed feedback proves 
particularly damaging. In connected 
classrooms, display of aggregated 
student results occurs shortly after 
students submit their individual work. 
Displays can give powerful clues to 
what students are doing, thinking, and 
understanding (Roschelle, Penuel, & 
Abrahamson, 2004). The teacher has 
immediate information and can use this 
to adjust instruction. Assignments can 
be smaller but more frequent — for ex- 
ample, requests to answer a question, 
solve a problem, state a position, write 
an equation, give a reason, participate 
in a simulation, or play a learning 
game (Stroup et al., 2002; Wilensky 
& Stroup, 2000, 2002). If the teacher 
chooses, aggregate class results can 
be displayed, students receive im- 
mediate private and non-threatening 
feedback in a way that encourages 
reflections and classroom discourse 
(Abrahamson, Davidian, & Lippai, 
2002; Dufresne, Gerace, Leonard, 
Mestre, & Wenk, 1996). 


Researchers have found that in a 
connected classroom, students are 
more likely to actively engage in the 
learning process. Student attitudes and 
motivation improve as teachers focus 
on conceptual understanding. Quick 
attention to alternative understand- 
ing changes the student experience 
in science and mathematics from 
memorization of large quantities of 
facts to understanding the relation- 
ships between concepts (Dufresne et 
al., 1996). 



In connected classrooms, 
display of aggregated 
student results occurs 
shortly after students 
submit their individual 
work. 

Present educational policy focuses 
on inputs to the educational system — 
students, teachers, resources, man- 
agement, standards, and preparation 
for high stakes test. The assumption 
behind the input argument rests with 
the logic that changing these inputs 
will change the desired outputs — in- 
creased student achievement, strategic 
learners, highly qualified workers, and 
contented teachers. The pedagogy of 
the connected classroom addresses a 
significant component of what teachers 
and students do inside the black box 
of their classroom (Black & Wiliam, 
1998). Inquiry and problem-solving 
approaches often leave unanswered 
the question of what students have 
learned from these experiences. Teach- 
ers need tools to better understand 
their students ’ learning as they engage 
their students in learning experiences. 
While classroom discourse processes 
have been examined as catalysts for 


fostering student active cognitive en- 
gagement and vehicles for assessing 
student understanding, the connected 
classroom offers tools for data col- 
lection and display that support these 
efforts. These tools provide a window 
into a learner ’ s prior and present under- 
standings and feedback loops that sup- 
port teacher’s instructional decision 
making and monitoring. Both teachers 
and students learn during instruction. 
Through constant observation and 
monitoring for understanding, teach- 
ers and learners may alter the course 
of the learning experience at critical 
junctures in the learning process and 
impact outcomes. 

Two important research studies 
with connected classroom technol- 
ogy are currently underway in the 
United States: 1) the 5-year NSF- 
funded (TPC-0456 1 24, 2005) Teacher 
Learning of Technology-Enhanced 
Formative Assessment project by 
Leonard, Gerace, Beatty and Feldman 
at the Scientific Reasoning Research 
Institute, University of Massachu- 
setts Amherst, and 2) the 4-year, IES 
funded Classroom Connectivity in 
Promoting Student Achievement in 
Mathematics and Science Achieve- 
ment project by Owens, Abrahamson, 
Pape, Irving, and Demana at The Ohio 
State University (R305K050045). 
The Massachusetts team hopes to 
better understand teacher learning 
of technology-enhanced formative 
assessment, to understand effective 
and efficient methods of teacher pro- 
fessional development for connected 
classrooms, and to develop tools and 
techniques for the evaluation of teacher 
pedagogy in connected classroom en- 
vironments. The project at Ohio State 
is an interdisciplinary research study 
focused on teaching and learning of 
mathematics and science at the 7 th to 
1 0 th grade levels with a national sample 
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Figure 1 . 

Traditional versus connected classroom 

Traditional Classroom 

Teacher assigns work. Students complete and submit. Teacher evaluates and returns. Students see individual work. 



Networked Classroom 

Teacher assigns work. Students complete and submit to connected classroom network. Computer aggregates and displays. 
Students and teacher see aggregated class data as well as individual work. 


Time: minutes 



of teachers using the TI-Navigator 
connected classroom hardware. This 
project includes a randomized cross- 
over trial design where the control 
group is sequentially exposed to the 
intervention. The project will measure 
student achievement in algebra 1 and 
physical science, student motivation, 
student dispositions toward mathemat- 
ics and science and self-regulated 
learning over a four year period for 
up to 145 teachers and their students. 
Both of these projects promise to offer 
scientifically -based evidence to further 
support the claims made for increased 



student motivation, engagement, and 
achievement in connected classroom 
environments. 

Conclusions 

Educators, policy makers, parents, 
and community members wish to 
maximize student achievement in the 
United States in science, mathematics 
and many other disciplines. To achieve 
this goal, both assessment of learning 
and assessment for learning must 
play important roles. Computer-based 
statewide testing programs, reliable 
and valid measures of student learning, 


and comprehensive data management 
systems that facilitate the flow of infor- 
mation between stakeholders support 
the goal of assessment of learning. 
Assessment programs sensitive to the 
needs of students in both the regular 
and special education populations 
provide insight into student learning 
over time and begin our efforts toward 
assessment for learning. Innovations 
such as connected classroom technolo- 
gies, however, offer greater potential 
for improved foimative assessment for 
classroom teachers on the front lines of 
assessment/or learning. In both cases 
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the issue resolves to using reliable 
data to make decisions, whether at the 
state, district, or school level, or the 
classroom teacher level. Educational 
technology provides a powerful tool 
to gather, organize, and analyze data. 
Teachers and policy makers need the 
expertise to responsibly use these data 
to benefit all our students. 
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